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1 Self-Organization of Networks 

Self- organization denotes "the process by which individual subunits achieve, through their coop- 
erative interactions, states ch aracterized by new, eme rgent properties transcending the properties 
of their constitutive parts." (jBiebricher et all Il995l ) Whether these emergent properties occur 
or not depends of course not only on the properties of the subsystems and their interactions, 
but also on suitable external conditions, such as global boundary conditions, the in/outflux of 
resources (free energy, matter, or information). A description that tries to include these condi- 
tions is given by the following heuristic definition: "Self-organization is defined as spontaneous 
formation, evolution and differentiation of complex order structures forming in non-linear dy- 
namic systems by way of feedback mechanisms involving the elements of the systems, when these 
systems have passed a critical distance fro m the statical equilibrium as a result of the influx of 
unspecific energy, matter or information." (jTeichmann and Wilkd . 119961 ) 

In this sense, the self-organized structure formation can be considered as the opposite of a hier- 
archical design of structures which basically proceeds from top down to bottom: here, structures 
are originated bottom up, leading to an emer ging hie r archy , where t he str uctures of the "higher" 
level appear as a new quality of the system (|Darleyl . 1 19941 ; lHakenl . Il978l ). For the prediction of 
these global qualities from local interactions fundamental limitations exist which are discussed 
e.g. in chaos theory. Moreover, stochastic fluctuations also give unlikely events a certain chance 
to occur, which in turn affects the real history of the system. This means, the properties of 
complex systems cannot be determined by a hierarchy of conditions, the system creates its com- 
plexity in the course of evolution with respect to its global constraints. Considering that also the 
boundary conditions may evolve and new degrees of freedom appear, coevolutionary processes 
become important, and the evolution may occur on a qualitatively new level. 

The self-organization of network structures, i.e. the emergence of links between a set of nodes 
is of crucial importance in many different fields. In electronic engineering, for instance, one 
is interested in the self- assembling and self-r epairing of electronic circuits (jCui and Lieber . 
200 ll : iMange and Staufferl . 11994 ; IWana. 120011). while in biology mod els for the self-wiring of 
neuronal networks are investigated (jSegev and Ben-Jacobl . Il998l . |2000| ). On the social level, the 
self-organization of human trail networks between different destinations is a similar problem 
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(jHelbing et all 119971 ). Also the establishment of connections on demand in telecommunication 
or logistics is related to the problem discussed here. 

In general, the self-organized formation of a network is rather different from drawing lines 
between a set of nodes. Often no a priori information about the network structure is provided, 
i.e. the network has to self-organize itself not only regarding the links but also regarding the 
nodes. This is the case for instance if the nodes to be linked to the network are "unknown" in 
the sense, that they first have to be discovered and only then can be connected. 

A common biological example is the formation of a trail system in ants (cf also Sees. 6 , 7) to 



connect a nest to a set of food sources that first have to be found (jCrist and Haefnerl . Il994l ; 



Schweitzer et all 119971 ). Such networks are known to be rather flexible and adaptive. After 
food sources are exhausted, they are "disconnected" from the existing network, because they 
are no longer visited and the respective trail is no longer maintained, but newly found food 
sources can be linked to the existing network as well. Hence, adaptivity is a desirable feature 
of self-organized networks. This means that new nodes can be linked to the existing network or 
linked nodes can be disconnected from the network if this is required, e.g. by the change of some 
external conditions. 

Noteworthy, such a behavior should be not governed by a "supervisor" or "dispatcher" , it should 
rather result from the adaptive capabilities of the network itself. In the self-wiring of neural 
structures, for example, a neuron grows from the retina of the eye towards the optic tectum 
(or superior colliculus) of the brain, without "knowing" from the outset about its destination 
node in the brain. Hence it has to navigate through an unknown environment in order to detect 
and to reach the appropriate area. It is known that gradients of different chemical cues play a 
considerable role in this navigation process. They provide a kind of positional information for 



the navigation of the growth cones (jGiererl . Il983l ). But in the very beginning, this positional 



information has to be generated interactively, and only in later stages may lead to established 
pathways. 

This points to the questions that shall be discussed in this chapter: Is it possible to link a set of 
nodes without using preexisting positional information or any kind of long-range attraction of 
the nodes? Can the process of generating positional information, i.e. the detection of "unknown" 
nodes and the estabishment of chemical gradients, and the process of network formation, i.e. 
the establishment of links between nodes, occur in parallel, on a comparable time scale, as a 
process of co-evolution? 

In the following, we will discuss a model where the generation of relevant information for estab- 
lishing the links between nodes results from the interaction of many agents, i.e. subunits of the 
system that are capable of performing some activities. Their collective interaction is based on 
(indirect) communication, which also includes memory effects and the dissemination of infor- 
mation in the system. The relevant ("pragmatic") information that leads to the establishment 
of the links then emerges from an evolutionary interplay of selection and reamplification. In the 
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next section, we will introduce the general framework for modeling such processes, while in Sees. 
3-7 two different applications of the model will demonstrate its suitability. 



2 Brownian Agents 

Recently, different computer architectures in distributed artificial intelligence have been devel- 
oped to simulate the collective behavior of interacting agents (cf. for instance the Swarm project 



at http://www.swarm.org/). However, due to their rather complex simulation facilities many 



of the currently available simulation tools lack the possibility to investigate systematically and 
in depth the influence of specific interactions and parameters. Instead of incorporating only as 
much detail as is necessary to produce a certain emergent behavior, they put in as much detail 
as possible, and thus reduce the chance to understand how emergent behavior occurs and what 
it depends on. 

Therefore, it would be feasible to have multi-agent systems (MAS) that can be also investigated 
by means of analytical methods (from statistical physics or ma thematics) - in ad dition to their 



computational suitability. The concept of Brownian agents (jSchweitzerl . 120031 ) is one of the 
possible approaches to serve for this purpose. It denotes a particular class of agents that combines 
features of reactive and reflexive agent concepts. A reflexive agent has an (internal) model or 
at least some knowledge about its environment that allows its to draw conclusions about some 
certain actions in a given situation. The reactive agent, on the other hand, simply "reacts" to 
signals from the environment without referring to internal knowledge. This signal-response type 
of action implies the same reaction to the same signal. From this perspective, the reactive agent 
is a rather simplex or minimalistic agent, while the reflexive agent is a rather complex agent. 



The Brownian agent (jSchweitzerl . 120031 ) is more advanced than the reactive agent but not yet of 



(k) 

the internal complexity of the reflexive agent. It can be described by a set of state variables u\ . 
The index i = 1, ...,7V refers to the individual agent i, while k indicates the different variables. 
These could be either external variables that can be observed from the outside, or internal 
degrees of freedom that can only be indirectly concluded from observable actions. Important 
external variables are = Tj, which denotes the space coordinate (mostly a vector in the 

(2) 

two-dimensional physical space), or u- = Vi, which is the individual velocity in the case of a 
moving agent. Both are assumed as continuous variables. 

The internal degrees of freedom, on the other hand, cannot be directly observed. They could 
be continuous or discrete variables. For instance, the state variable u\ = 9i £ {—1,0, +1} may 
distinguish between three different responses to certain environmental conditions or to incoming 
information. For example, agents with 6 = — 1 may not be affected by a particular signal, while 
agents with 9 = +1 may respond to it. An important continuous state variable in the context 
of Brownian agents is the internal energy depot uf^ = e^, which determines whether agent i 
may perform a certain action or not. This includes the assumption that all actions - be it active 
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motion or communication or environmental changes - need to use "energy". In general, this 
term describes not just the physical free energy that is dissipated e.g. during active motion, it 
intends to cover also other resources needed to perform a certain action. 

Noteworthy, the different (external or internal) state variables can change in the course of time, 
either due to impacts from the surrounding, or due to an internal dynamics. Thus, in a most 
general way, we may express the dynamics of the different state variables as follows: 

A W 

au i A k ) _|_ -r-stoch /-in 

dt h + 1 K) 

For the Brownian agents, it is assumed that the causes for the temporal change of Ui may 
be described as a superposition of deterministic and stochastic influences, imposed on agent i. 
This picks up the ingenious idea first used by Langevin in early 1900 to describe the motion of 
Brownian particles - and is basically the reason why this agent concept is denoted as Brownian 
agent. A Brownian particle moves due to the impacts of the surrounding molecules whose motion 
however can be observed only on a much smaller time and length scale compared to the motion 
of the Brownian particle. Thus, Langevin invented the idea to sum up all these impacts in a 
stochastic force with certain statistical properties. 

For the Brownian agent, we will exploit Langevins idea in a similar manner, i.e. we will sum 
up influences which may exist on a microscopic level, but are not observable on the time and 
length scale of the Brownian agent, in a stochastic term J-f tocil , while all those influences that 
can be directly specified on these time and length scales are summed up in a deterministic term 

(k) 

/• . Such a distinction basically defines the level of coarse-grained description for the multi- 
agent system. The "cut" may prevent us from considering too much "microscopic" details of the 
MAS, while focussing on particular levels of description. The summed up stochastic influences 
might result from a more fine-grained deterministic description - but instead of taking this into 
detailed account, just some specific statistical (gross) properties are considered on the coarse- 
grained level. Notheworthy, the strength of the stochastic influences may also vary for different 
agents and may thus depend on local parameters or internal degrees of freedom. 

The deterministic part contains all specified influences that cause changes of the state 
variable . This could be non-linear interactions with other agents j G N, external conditions 
- such as forces resulting from external potentials, or the in/outflux of resources etc. or even an 
eigendynamics of the system that does not depend on the action of the agents. In the example 
of an ecosystem this eigendynamics may describe day/night or seasonal cycles, or the agent- 
independent diffusion of resources within the system. 



3 Basic Agent Model of Network Formation 

In order to set up a Brownian multi-agent model for the formation of networks we need to specify 
(i) the relevant state variables u- fc ^ and (ii) the dynamics for changing them, i.e. ii^. For the 
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application considered, we may assume that each Brownian agent is characterized by three state 
variables: spatial position r», (discrete) internal state 9i and (continuous) internal energy depot 
ej. For the change of the agent's position we may assume an overdamped Langevin equation: 



dt 



fi + V2Fiti(t) 



(2) 



The second term denotes the stochastic influences, where is white noise with (&(£)) = 
and £j(*0) = $ij ~ *')■ The strength of the stochastic force £j could be in general an 
individual parameter to weight the stochastic influences, this way it can for example measure 
the individual sensitivity rji oc of the agent. The deterministic influences in the first term, 
Si, are in the considered case assumed to result from the the (direct or indirect) interaction 
between the agents, as specified below. 

The internal degree of freedom 9i{t) should have one of the following values: 6% G {0, — 1,+1}. 
Initially, 9i(to) = holds for every agent. The variable 0i can be changed in the course of time 
by an interaction between the moving agents and the nodes. To be specific, we consider a two- 
dimensional surface, where a number of j = 1, ...,z nodes are located at the positions r z - (cf. 
Fig. [[]). A number of z + nodes should be characterized by a positive potential, Vj = +1, while 
z_ = z — z + nodes have a negative potential, Vj = —1. We note explicitely, that the nodes 
do not have any long-range effect on the agents, such as attraction or repulsion. Their effect is 
restricted to their location, rf . 



Figure 1: Example of a regular distribution of 40 nodes on a lattice of size A = 100 x 100. 
For the computer simulations, periodic boundary conditions have been used. z+ = 20, z_ = 20. 
- x| indicates nodes w i th a potential Vj = —1, {+} indicates nodes with a potential Vj = +1. 
( Schweitzer and Tilch . 2002h 



It is the (twofold) task of the Brownian agents, first to discover the nodes and then to link nodes 
with an opposite potential, this way forming a self-organized network between the set of nodes. 
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If an agent hits one of the nodes, its internal degree of freedom is changed due to the following 
equation: 

Afc(t) = f J 6 { r J ~ r ^)) dr ( 3 ) 

j=l J A 

The delta function is equal to 1 only for r z - = n and zero otherwise. So, eq. §3§ indicates, that 
an agent can change its internal state, 9i, only if it hits one of the nodes. Then it takes over the 
value of the potential of the respective node, Vj, which means 6i remains constant if Vj = 6i, 
and 0{ — > Vj, if Vj We note that the probability for a (pointlike) agent to hit a (pointlike) 
node is almost vanishing. However, the computer simulations discussed in the following section 
are carried out on a discrete lattice, so the agent and the node both have a finite extension, in 
which case Eq. [3] makes sense. 

Eventually, we have to describe the dynamics of the internal energy depot, ej, of the agent. In 
this specific application, it is assumed that every time t l n+ , t % n _ when Brownian agent i hits one 
of the nodes n+ or n— its internal energy depot e, is filled up to a maximum value e max . Then, 
the agent is in an active state that allows for certain actions. In our case, the agent is able to 
produce a chemical, either component (—1) or (+1). Which kind of chemical is produced depends 
on the actual value of the internal parameter 0i(t) of the agent as specified below, whereas the 
amount of chemical produced depends on the actual value of the internal energy depot ej(t). 
For simplicity, we assume for the amount of the production rate 

Sioca -> Si(9i,t) = (3 ei(t) (4) 

The internal energy depot can only be charged when hitting one of the nodes and can be filled 
only to a maximum 6 max - On the other hand, it is decreased by the production of the chemical. 
Hence, in the general balance equation 

^ = qi(ri,t)-pi(r h t) (5) 
the space and time dependent influx of energy into the internal depot is given by: 

qi{r h t) = (e max - e<(t)) b{r) - n)6(t - ? n ±) (6) 
wheras the "outflux" of energy from the agent's depot, pi{ri,t), reads accordingly: 

Pi(ri,t) = 8i(9i,t) (7) 



We note that in previ ous investigations (jEbeling and Schweitzerl . 120031 : lEbeling et all Il999l : 



Schweitzer et all Il998l ) an additional term for the active (accelerated) motion of the agent was 
included in Eq. [71 Further, a dissipative loss, — cej, of the internal energy depot was considered. 
Both terms are neglected here. 
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Considering Eq. Q, we can deduce from Eq. (|5|) the time-dependence of the production rate 
as Si(9i,t) ~ exp{— fit}. The dependence on 0j, however, involves specific assumptions of our 
network model: 



Si(0i,t) 



(1-0;) exp{-/3 (*-£_)} 



(8) 



Eq. (JHJ) means that the agent is not active, as long as 6% = 0, which means before it hits one of 
the nodes the first time. After that event, the agent begins to produce either component (+1) 
if 0i = +1, or component (—1) if Q% = —1, at a rate that exponentially decreases with time. 
The maximum production rate s max depends on the maximum amount e max , after the internal 
energy depot was charged at one of the nodes. 

The spatio-temporal concentration of the chemicals shall be described by a chemical field hg(r, t) 
consisting of the two components (+1) or (—1), which obeys the following dynamics: 



dh g (r,t) 
Ot 



N 

-k h hg(r,t) +^2si(6j,t) 
i=i 



6(r-ri(t) 



(9) 



The first term describes the exponential decay of the existing concentration due to spontaneous 
decomposition of the chemical, where is the decomposition rate. The second term denotes 
the production of the field by the agents. Here, 5g l g i means the Kronecker Delta used for discrete 
variables, indicating that the agents only contribute to the field component that matches their 
internal parameter 0j. The Delta function S(r — Vi(t)) means that the agents contribute to 
the field only locally, at their current position, r,;. Diffusion of the chemical substances is not 
considered here. 

As a last step, we have to specify how the existence of the two different chemicals affect the 
motion of a Brownian agent, described by Eq. ([2]). We assume that the interaction term /; results 
from an indirect communication among the agents. By producing one out of two chemicals at 
its current position, each agent generates local information that is stored outside the agent, i.e. 
the environment acts as a kind of external memory. This information should be accessible to 
other agents in the vicinity. Specifically, we assume that the agent is sensitive to local gradients 
in these two chemical components dependent on its current internal state Q{\ 



t 1 



(1 + 0* 



Oh^(r,t) 



Or 



(1 



dh + i(r,t) 



Or 



(10) 



Here, a is a dimensional constant only, but it could als o act as an individ ual response parameter 
weigthing the importance of the information received (jSchweitzerl . 119971 ). 



This way, by producing and responding to two different local information in the system, the 
Brownian agents communicate with each other. The chemical field plays the role of an order 
parameter. It does not exist from the outset, but is generated by the agents. Once it is established, 
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Figure 2: Circular causation between the Brownian agents in the different internal states, C_i, 
C+i, and the two-component chemical field, hg(r,t) 

it reduces the "freedom" of the moving agents by guiding their motion using the information 
generated by other agents. 

Fig. [2] summarizes the non-linear feedback between the two-component chemical field and the 
agents, as given by the Eqs. ([8]), (|10p . Our model assumes, that agents with an internal state 
0i = do not contribute to the field and are not affected by the field. They simply move like 
Brownian particles. Agents with an internal state #j = +1 contribute to the field by producing 
component +1, while they are affected by the part of the field that is determined by component 
— 1. On the other hand, agents with an internal state 9i = — 1 contribute to the field by producing 
component —1 and are affected by the part of the field, which is determined by component +1. 
Moreover, if the agent hits one of the nodes, the internal state can be switched due to Eq. ([3j). 
Hence, the agent begins to produce a different chemical, while affected by the opposite potential. 
Precisely, at one time the agent does not respond to the gradient of the same field, which it 
contributes to via producing a chemical. 

As the result of this nonlinear feedback between the Brownian agents and the chemical field 
generated by them, we can observe the selforganized formation of a network shown in the 
following section. 



4 Computer Simulations of Network Formation 



For the computer simulations, a triangular lattice with periodic boundary conditions has been 
used. The agents start initially at random positions. For the evolution of the network, we evaluate 
the sum h(r,t) of the two field components generated by the agents. For the plots, however, we 
have to match these values with a grey scale of 256 values, which is defined as follows: 



c(r,t) 
h(r,t) 



255 



1 - log 1 + 



h(r,t) - h min (t) 

^max(^) ^min(^) 



h + i(r,t) + h-i(r,t) 



11 
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This means that the highest actual value, h mauX {t), always refers to black (c = 0), whereas the 
actual minimum value, h m i n (t) encodes white (c = 255). Both extreme values change of course 
in time, therefore each snapshot of the time series presented has its own value mapping. 

(a) (b) (c) (d) 






Figure 3: Formation of links between 4 nodes: (a) initial state, (b) after 100, (c) after 1.000, (d) 
after 4.500 simulation ste ps. Lattice size 30 x 3 , 450 agents. Parameters: s max = 25.000, kh = 
0.01, (3 = 0.2, k h = 0.01. dSchweitzer and Tilchl . l2002h 



As a first example, we show the evolution of the connections between four nodes. Fig. [3] would 
suggest that in the course of time all nodes with an opposite potential should be connected. This 
however is not true because the existing connections cause a screening effect, which forces the 
agents to move along existing connections rather than making up new ones. This screening effect 
becomes more obvious, when the number of nodes is increased. Fig. 0] shows the time evolution 
of a network which should connect 40 nodes, as shown in the setup of Fig. [TJ Here, we see that 
in the course of time the agents aggregate along the connections, which results in higher agent 
concentrations and in higher fields along the connections 
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Figure 4: Time series of the evolution of a network after 10, 100, 1.000 simulation steps. Lattice 
size 100 x 100, 5.000 agents, 40 nodes , z + = 20, Z- = 20. Parameters: s max = 10.000, k h = 0.03, 
(3 = 0.2. (jSchweitzer and Tilchl . l2002h 



In Figs. l3|41 the connections between the nodes exist as a two-component chemical field generated 
by the Brownian agents. This self-assembling network is created very fast and remains stable after 
the initial period. We note that the network formation is not restricted to regular or symmetric 
distributions of nodes. Fig. [5] shows a simulation, where different nodes are connected with a 



A video of these computer simulations can be found at http: //intern, sg . ethz . ch/f schweitzer/until2005/network .html 
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center. An extension of the model has been applied to simulate th e trunk trail formation in ants 
connecting a nest to different food sources (jSchweitzer et aull997l ) and will be discussed in Sees. 
6, 7. 




Figure 5: Formation of links between a center (z_ = 1) and surrounding nodes (z+ = 7) after 
10.000 si mulation steps. Lattice size: 5 x 50, 2.000 agents. Parameters: s max — 20.000, k h = 0.02, 
(3 = 0.2. (jSchweitzer and Tilchl . l2002h 



Patterns like the networks shown in Figs. 13)4)51 are intrinsically determined by the history of 
their creation. It means that irreversibility and early symmetry breaks play a considerable role 
in the determination of the final structure. These structures are unique due to their evolution, 
and therefore can be understood only within a stochastic approach, as provided for example by 
the Brownian agent approach. 



5 Estimation of the Network Connectivity 

In order to characterize a network, one of the most important questions is, whether two nodes 
k and I are connected or not. In the model considered, a connection is defined in terms of the 
chemical field h(r,t) produced by the agents. In the beginning, however, the agents randomly 
visited almost every lattice site before their motion turned into a bound motion between the 
nodes. Therefore, the field h(r,t) has a non-zero value for almost every r, which exponentially 
decays, but never vanishes. Hence, in order to define a connection in terms of h(r,t), we have 
to introduce a threshold value h^, which is the minimum value considered for a link. More 
precisely, a connection between two nodes k and / should only exist, if there is a path s £ A 
between k and I on which the actual value of the field is larger than the treshold value: 

h(s,t)>h thr fors^A (12) 
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Such a definition does not necessarely assumes that the connection has to be a direct link. 
Instead, it could be any path s, which may also include other nodes, as long as the value h(s,t) 
along the path is above the threshold. 

We want to define the local connectivity Eik as follows: 

1 if /c and I are connected by a path s € A, 
Eik = { along which h(s,t) > /i t hr (13) 
otherwise 

We note that the connectivity Ey. does not change, if two nodes k and I are connected by more 
than one path. 

If we consider a number of z nodes, then the global connectivity E which refers to the whole 
network, is defined as follows: 



E 



fc=l l>k 
k=l l>k 



z z 



Dependent on the configuration of nodes, there may be numerous different realizations for the 
connections, which result in the same connectivity E. 

In order to use the definition for the connectivity to evaluate the simulated networks, we first 
have to define the threshold value fothr- This should be the minimum value of h(r,t) along a 
stable connection between t wo nodes. Based on this definition, we have derived in a recent paper 



Schweitzer and Tilchl (|2002l ) an estimate for h t hr'- 



s ( s ■ \ 1/4 

7 -•'max / •'mm \ /-ir\ 
frthr = n —j— ( I ( 15 ) 

Provided the set of parameters, used for the simulations, we find for the threshold the value 
^thr = 1-7 x 10 4 , which is approximately /i tm . ~ 2s max - We note, that this is an estimate, which 
might give a rather high value and thus ensures, that values for h(s, t) above the threshold really 
represent a stable connection s. 



In a recent paper ISchweitzer and Tilchl ([2002), we have investigated how the global connectivity 
E evolves in the course of time and how it changes in dependence on the density of agents, n. 
In agreement with the visible evolution of the network presented in Fig. HI three different stages 
can be clearly distinguished: (i) an initial period (t < 10 2 ), where no connections yet exist, 
(ii) a transient period (10 2 < t < 10 4 ), where the network establishes, (iii) a saturation period 
[t > 10 4 ), where almost all nodes are connected, and only small fluctuations in the connectivity 
occur. 

Here, we focus on the question, how the connectivity changes in dependence on stochastic in- 
fluences. According to Eq. ([!]), in the Brownian agent approach the different state variables of 
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the agent are changed both by deterministic and stochastic influences. In our agent-based model 
of network formation, stochasticity influences are explicitely considered only in the equation of 
motion of the Brownian agents, Eq. (|2j). There, it was assumed that the strength of the stochas- 
tic force £j could be in general an individual parameter. This idea will be used in Sec. 6, here 
we want to assume for the moment that e% = e is equal for all agents and can be further related 
to the temperature T, which is a physical measure of fluctuations in the system, independent 
of the agents. It was Einstein, who derived in 1905 a relation between the temperature and the 
diffusion of simple Brownian particles, D = (&b/7o)T ', where the proportionality is given by the 
Boltzmann constant ks and the friction coefficient 70. Identifying e = D, which is correct for the 
physical interpretation of the overdamped Langevin Eq. (J2J), we can now express the strength 
of the stochastic force by the temperature T, which as a global parameter acts as a boundary 
condition for the dynamics of the multi-agent system. 

From the physics perspective, network formation - as other p rocesses of dissipative structure 



formation in non-equilibrium systems (jBiebricher et all 119951 ) - should become possible only 



•_pr 

3) 



below a certain critical temperature T c . Above T c , fluctuations in the system rather destroy the 
structures emerging. With respect to Eq. ([2]), this means that for T > T c , the agents cannot 
pay sufficient attention to the determinstic influences given by /j and thus behave more or less 
as simple Brownian particles. The delicate balance between the stochastic and deterministic 
influences of course depend on the current value of fi, which in turn results from the history of 
the indirect communication process described by Eqs. Q, (jlOp . However, in a so-c alled mean- 
field approximation for the agent density and the dynamics of the field components (jSchweitzerl . 



20031 ) we were able to derive an expression for the critical temperature T c below which the 



network formation becomes feasible: 

- 2 *^ < 16 > 

This equation relates T c to the me an density of age nts, n = N/A, and an average production 
rate of the chemicals s, derived in (jSchweitzerl . 120031 ) as: 

_ Smax ^max (17) 

In S m ax ffi S max 

By varying the stochasticity, expressed by the parameter T relative to the reference value T c , we 
are now able to investigate the connectivity by means of computer simulations, carried out for 
the same setup as used for Fig. [H Fig. shows the connectivity, averaged over 200 simulations, 
as a function of the reduced temperature T/T c , whereas Fig. [7]shows some simulated realizations 
of the networks for different reduced temperatures. 

Fig. [5] clearly indicates that an optimal range of temperatures, 0.3 < T/T c < 0.5, exists where 
the average connectivity reaches a maximum. For the given setup of nodes and the given set 
of parameters, within this optimal range in the average more than 90 percent of the nodes are 
connected. For T > 0.5 T c , the connectivity breaks down drastically, which means that the 
motion of the agents is mainly determined by the stochastic forces, and no attention to the 
gradient of the field is payed. 
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Figure 6: Network connectivity E, Eq. (|14|) . a veraged over 200 s imulations vs. reduced temper- 
ature T/T c Eq.( dS]). For the setup see Fig. H (jSchweitzerl . booah 
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Figure 7: Network formation dependent on the reduced temperature T/T c , Eq. ffT6 |) . Shown is 
the v alue c(r,t), Eq. (fTTj) after 5.000 simulation steps. Other parameters see Fig. HI (jSchweitzerl . 
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On the other hand, for T < 0.3 T c , the connectivity decreases because during the transient pe- 
riod of establishing the network, the agents have payed too much attention to the gradient, and 
therefore are trapped in the first established links, instead of moving around. In this range of 
temperatures, the average connectivity (E), Fig. 03 displays large fluctuations, so almost every 
realization between E = 0.5 and E = 1 has a certain likelyhood. Here, the early history in creat- 
ing the network plays an important role for the eventual pattern and the connectivity. However, 
in the optimal range of temperature, the effect of the early symmetry breaks is weakened to a 
certain extent, thus leading to smaller fluctuations in E. 

We also want to note, that in the range of smaller temperatures, of course less connections 
between the nodes occur, but these connections provide a much stronger link in terms of the 
field h(r,t). One of the reasons for the breakdown of the connectivity for T > 0.5T C in Fig. [5] 
results from the fact, that many of the links established have a value below the threshold, ftthrj 
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and therefore are not considered as stable connections. This can be seen by comparing the 
pictures in Fig. [7J Here, the nodes still seem to be connected for T = 0.6 T c , the value of the 
related field however is below the threshold, /ithr- 

6 Modeling Trail Formation in Ants 

In this section, the network model introduced above is extended to simulate the formation of 
directed trails, as observed in group-raiding ants. So, the nodes are given now by a nest and 
different food sources, and the links are given by the directed trails between nest and food. 
These connections do not exist from the very beginning; they have to be generated by the 
agent community in a process of self-organization. Hence, the agents have to perform two quite 
different tasks, which are referred to each other: first, they have to detect food places - unknown 
to them, and then they have to link these places to their original starting point by forming a 
trail, with local chemical orientation as the only tool provided. 

For our model, we assume that initially all agents are concentrated in one place, the "nest". 
The internal parameter, 0j = ±1, describes whether an agent has found food yet or not. If the 
agent starts from the nest, the internal parameter is always set to 9i = +1. Only if the agent has 
detected a food source, 9i is changed to 0j = — 1. Dependent on the value of 6{, the agent is able 
to produce one of two different chemicals with a rate Si(9i,t), which was already described in 
Eq. ([8]). I.e., the agent produces chemical of component (+1), when it leaves the nest (0j = +1) 
at time t l n , and starts to produce chemical of component (—1) instead, after it has found food 
(9i = —1) at time t\. The dynamics of two chemical components is again given by Eq. Q. 
Further, the agents are affected by the different kind of chemicals as described by Eqs. (j2)) (fTQl) . 

On could argue that for the simulation of foraging trails of ants, the most simple assumption 
could be that only individuals which found food create a pheromone trail, while turning straight 
back to the nest. These trails could then be used by other ants to find the food sources, once 
they found the trails. However, those simulations have to overcome the problem where the 
first ant which found food got the information from to find its way back home. In biological 
systems, geocentric or egocentric navigation could provide this additional information needed 
for successfully returning to the nest. Since our model simulates the formation of trunk trails 
without counting on navigation and internal storage of information, our main assumption is 
that a simple chemotactic response would be sufficient to generate trails between the nest and 
the food sources. However, we have to distinguish trails which have been created during the 
(unsuccessful) search for food from trails which really lead to a food source. In our model, this 
additional information is encoded by assuming that the agents produce, and respond to, two 
different chemical fields. 

For the computer simulations, we have to address an additional problem that results from 
obvious differences in complexity between biological entities and Brownian agents. It is known 
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from central foragers like ants that they are able to leave a place where they don't find food 
and reach out for other areas. This indicates that they have at least a certain ability to increase 
their mobility since they have a certain aim, finding food. Agents in our model however do not 
have aims and do not reflect a situation. So, they stick on their local markings even if they they 
did not find any food source. In order to increase their mobility in those cases, we assume that 
every agent is described by an additional internal variable, its individual sensitivity rji to follow a 
chemical gradient. As long as the agent does not hit a food source, this sensitivity is continuously 
decreasing via an increase of the stochasticity in Eq. ([2]), i.e. we assume here rji = ct/ej, where 
a is the response parameter to weight the influence of the gradients. Increasing the individual 
stochasticity then means that the agent more and more ignores the chemical detected and thus 
becomes able to choose also sites not visited so far. 

However, if the agent does not find any food source after a certain number of steps, the sensitivity 
r\i becomes negligible, and the agent only behaves like a Brownian particle. From the utilitarian 
perspective, such a agent becomes "useless" for the exploitation of food sources, if its individual 
noise level, £j, exceeds a maximum value, e max (which could be related to the critical temperature, 
discussed above). So we simply assume that the agent "dies" at this level and is removed from 
the system. On the other hand, if the agent hits a food source, its sensitivity is set back to the 
initial high value and is kept constant to increase the chance that the agent finds its way back 
along the gradient. The switch of the level of sensitivity is accompanied with the use of the tw o 



different chemical markers, and in this particular sense reminds on chemokinesis (jDunnl . Il983l ). 
where the level of activity can be changed due to chemical conditions in the surrounding. 

Using the internal parameter 8i again, we may assume that the sensitivity rji of the agent changes 
according to: 

^=^(*) = |{(l + ^b + <fe(i-4)l -(l-0i)e o \ (18) 
W{t) 2 1 L J J 

Here, 5e is the increase of the individual noise level, per time step if the agent has not found 
any food source. 

This way, we have completed our model for the trail formation in ants. The basic ingredient 
is the chemotactic response to a chemical gradient which has been produced by the agents 
themselves. Hence, the basic equations are given by the equation of motion for the agents, 
Eq. (|2j), which is coupled with the equations for the two chemical fields, Eq. Q. We note that, 
with respect to biology, there are different parameters which may influence trail following in 
addition to sensitivity, such as trail fidelity, traffic density, detect i on distance, endurance o f 



the trail, navigation capabilities etc. (jEdelstein-Keshet et all Il995l : lHaefner and Cristl . 1 19941 ). 



Contrary, our model considers only minimal assumptions for the trail formation. Here, the 
formation of trail patterns is solely based on simple local chemical communication between the 
agents, with no additional capabilities of orientation or navigation. The computer simulations 
outlined below should prove that the simple local rules assumed for the action of the agents are 
sufficient enough to solve such a complex problem. 
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7 Computer Simulations of Trail Formation in Ants 

Different from arbitrary non- directed tracks which are commonly used by the agents but lead to 
nowhere, directed trails should link a starting point (e.g. a nest) to a destination point (e.g. a food 
source) and back. In order to define start and destination, we assume on the two-dimensional 
surface a center (nest) where the agents are initially concentrated, and random distribution 
of five separated food sources food sources unknown to the agents. As in the network model 
described before, both the nest and the food sources, do not attract the agents by a certain 
long-range attraction potential, they are just particular sites of a certain size (defined in lattice 
sites). It is assumed that the food sources can be exhausted by the visting agents, which are 
assumed to carry part of them back to the nest. 

Fig. [8] shows a time series of the trail formation in terms of the spatial concentration of chemical 
component (—1), which is coded again in a grey scale. In Fig. [8^, we see that two of the food 
sources randomly placed on the lattice, have about the same distance to the nest, but the one, 
which - by chance - has been discovered first, will also be the one linked first to the nest. This 
reflects the influence of initial symmetry breaking effects. 

If the food sources are exhausted by the agents carrying food to the nest, they vanish. Therefore, 
the individual sensitivity of the agents coming from the nest to the food cannot be set back to the 
high initial value and the agents further increase their mobility by ignoring the trail, they reach 
out again, and by chance discover new food sources. But since a trail already exists, those sources 
in a close vicinity to the one that disappeared have a larger probability to be discovered. Part 
of the "old" trail is re-used then to link the new food source to the nest (compare Fig. [8k,b,c). 

Fig. [Hb shows that at the same time also more than one sources could be linked to the nest 
by trails. However, all trails compete for the agents to be maintained, and the trails could 
survive only if the concentration of the chemicals is above a certain critical value. Therefore, the 
coexistence of different trails only lasts for a short time. Hence, the trails to the different food 
sources will appear one after the other, with the old trails disappearing again by decomposition 
because they are no longer maintained. After a source is exhausted, we usually observe a time 
lag before the next one is exploited. This is needed for the system to "forget" about the useless 
trail and to establish a new trail to the next source. Thus, we can distuingish between two 
alternating dynamical stages: (i) a stage of rather random motion of the agents {exploration of 
the sources), and (ii) a stage of directed motion of the agents along the trail [exploitation of the 
sources). 

The flexibility of the model is indicated by the fact, that even after a long period of trail formation 
in the lower part of the lattice, the agents are able to link the food source on the upper left 
side of the lattice to the nest (Fig. [HJl) and therefore finally have detected and linked all five 
sources to the nest. The desorienation stage before exploiting the last source is especially long, 
because the agents could not build on any previous part of a trail into that direction, whereas 
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Figure 8: Formation of trails from a nest (middle) to five randomly placed food clusters. The 
distribution of chemical component (—1) (see text) is shown after (a) 200 0, (b) 4000, (c) 8500, 
and (d) 15000 simulation time steps, respectively. (jSchweitzer et all 119971 ) 



the desorientation stage before exploiting the third and the fourth source is very short, because 
of the trails which already exist into that area. 

8 Conclusions 



In this chapter, we have proposed a multi-agent approach to model the self-organization of net- 
works. This approach is based on the interactions of the agents on a local or "microscopic" level, 
which could lead to the emergence of the structure as a whole on the global or "macroscopic" 
level. In our model, the emergence of network structures, i.e., the self- organized formation of 
links results from the dynamic interaction of the agents, which is different from drawing lines 
between a set of known nodes. The agents have to solve two different tasks in parallel, namely 
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to discover new nodes and to link them to the network without external guidance. They also 
have to ensure the adaptivity of the network. 

In our model, the self-assembling of networks between arbitrary nodes is based on the nonlinear 
interaction of Brownian agents. Different from a circuit diagram, for instance, which determines 
the different links between the nodes in a top-down approach of hierarchical planning, the connec- 
tions here are created by the agents bottom-up. The spontaneous emergence of different network 
structures was characterized as a self-organizing process. As in every evolutionary game, also 
for the emergence of networks critical parameters exist. As an example, we have discussed the 
strength of fluctuations compared to the influence of the chemical gradients, but this also holds 
for other parameters in the model. If, for instance, the decay rate for the decomposition of the 
chemicals is too high, or the sensitivity of the agents is too low, they loose their local orientation 
and get lost, so that no links appear. If, on the other hand, the decay rate is too low, it takes a 
much longer time, before distinct links appear, because the selection pressure is too low. 

The model turned out to be very flexible regarding the geometry of the nodes to be connected. 
The locations of the different nodes act as a boundary condition for the structure formation, 
but do not determine the way of connecting the different nodes. If, for example, a particular 
link breaks down, the agents would be able to repair it by re-establishing the field, again, or 



by creating a new one. It was shown (jSchweitzerl . 120031 ) that the switching behavior between a 



connected and a disconneced state can be very short, which would allow the construction of a 
dynamic switch. 

To maintain a network, to keep it updated, to repair links, are ambitious aims for which natural 
paradigms exist, e.g. in social insects or neural networks. These have also inspired the design of 
our multi-agen t model. Compared to the complex "individual-based" models in ecology f DeAn- 



gelis and Gross, [I992|; iHuston et all 119881 ) the agent model proposed here provides a very simple 



but efficient tool to simulate a specific structure with only a view adjustable parameters. The 
major difference to biology is denoted by the fact, that the agents used in the simulations do not 
have individual capabilities to store and process informations about the locations of the nodes, 
or to count on navigational systems or additional guidance. They rather behave like physical 
particles whi ch respond to local force s in a quite simple manner, without "implicit and explicit 



intelligence" (jHaefner and Cristl . 119941 ) . Noteworthy, these local forces only result from the chem- 
ical markings produced by themselves. This way, we propose a kind of minimalistic approach 
for the formation of networks which is build on the dynamical essence of the feedback processes 
rather than on the internal complexity of the agents. 
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